Privacy Preserving Machine Learning: Related Work

A practical scenario of PPML is where only one central party has the entire data on which the ML algorithm has to be learned. Agrawal and Ramakrishnan [1] proposed the first method to learn a Decision Tree classifier on a database without revealing any information about individual records. They consider public model private data setting where the algorithm and its parameters are public whereas the data, over which the algorithm is trained, is kept private. The privacy of individual records in data is achieved through data perturbation. This notion of privacy was later identified as differential privacy by Dwork [2]. Though the data is private in the scenario considered by Agrawal and Ramakrishnan, where algorithm is trained on the data and only the learned algorithm is revealed, some orthogonal scenarios consider publishing the perturbed data thereby leading to public data setting with respect to the perturbed data. Chaudhuri and Monteleoni [3] proposed learning of Logistic Regression with differential privacy in which they consider the same setting of public model private data as above. However, a major distinction is that they perturb the learning algorithm rather than the data itself (unlike the method of Agrawal and Ramakrishnan) thereby not revealing information about individual data points upon which the algorithm is learned. Chaudhuri et al. [4] further generalized this solution for learning Support Vector Machine (SVM) under the same settings. Zhang et al. [5] gave a differential privacy solution to learning Logistic and Linear Regression models by perturbing the objective function similar to Chaudhuri and Monteleoni under the public model private data setting. Rubinstein et al. [6] came up with differentially private learning of SVM under public model private data setting, where differential privacy is guaranteed by introducing weight regularization in objective loss function. Jain et al. [7] proposed a mechanism of training Deep Belief Networks with differential privacy using drop-out (unlike the above methods of objective perturbation which rely on regularization for differential privacy). Like all the above differential privacy mechanisms, Jain et al. also consider public model private data setting.